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Executive Summary 


This SERC 2014-2018 Technical Plan is intended to provide the vehicle by which to align 
the SERC Vision and Research Strategy with the Sponsor's Core funding priorities. It 
describes the SERC Vision, the Sponsor's needs, and the SERC's response to these needs, 
supported by research thrusts in the four areas of Enterprises and Systems of Systems 
(ESOS), Trusted Systems (TS), Systems Engineering and Systems Management 
Transformation (SEMT) and Human Capital Development (HCD). A Grand Challenge 
statement is presented for each research thrust area, along with a strategy to address it. 
Eleven research programs have been identified to support this strategy. Research 
projects are then presented which support each of these programs, consisting of 
existing and future projects. 

The stated grand challenges are as follows: 

ESOS - Create the foundational SE principles and develop the appropriate MPTs 
to enable the DoD to architect, design, analyze, monitor and evolve complex 
enterprises and systems of systems to provide the DoD with an overwhelming 
competitive advantage over its current and future adversaries. 

TS - Achieve much higher levels of system trust by applying the systems approach 
to achieving system assurance and trust for the increasingly complex, dynamic, 
cyber-physical-human net-centric systems and systems of systems of the future. 

SEMT - Transform the DoD community's systems engineering and management 
methods, processes, tools and practices to enable much more rapid, concurrent, 
flexible, scalable definition and analysis of the increasingly complex, dynamic, 
multi-stakeholder, cyber-physical-human DoD systems and systems of systems of 
the future. 

HCD - Discover how to dramatically accelerate the professional development of 
highly capable systems engineers and technical leaders in DoD and the defense 
industrial base and determine how to sustainably implement those findings. 

The SERC 2014 - 2018 Technical Plan will deliver the greatest impact for DoD and the 
intelligence community (1C) by 

1. Conducting long-term research that makes significant progress on the grand 
challenges 
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2. Transitioning that research into practice within DoD, the 1C, defense industrial 
base, and other federal agencies; and by developing more powerful ways to 
facilitate such transition 

3. Amplifying sponsor resources by forging relationships with other organizations 
that become partners, contributing their resources and energy to the SERC and 
adopting SERC research 

4. Strengthening the existing SERC culture, mechanisms and focus on collaboration 

5. Instituting new approaches to educate future systems engineers and engineers 
that leverage the full strength and diversity of the collaboration 

1 SERC Vision 


At the end of 2018, after executing this Plan, the SERC will have the following 
characteristics: 

• The SERC has indeed become the go-to place for high-quality SE research and 
exploratory development. Its research is widely applied in DoD and the defense 
industrial base with tangible impact affecting billions of dollars of acquisitions; its 
research results are woven into the curricula of dozens of university programs 
(including many outside the set of SERC Collaborators) that are educating 
thousands of students. 

• The SERC has stabilized at 25 high-capability, collaborative universities. It 
includes ten of the US News and World Report top-25 
industrial/manufacturing/systems engineering departments. 

• SERC Collaborators graduate over half of the US MS-SE and PhD-SE graduates per 
year. Many PhD graduates join other SERC universities as faculty or staff, 
significantly increasing the breadth and depth of research collaborations. 
Collaborators attract and educate the best students, drawn from current DoD 
and defense industrial base employees and from those who are attracted to 
systems engineering by the vigor and quality of Collaborator educational 
programs. 

• The SERC provides much of the leadership in SE-related professional societies, 
increasing collaboration among them. It continues to operate and grow the 
Conference on Systems Engineering Research as the premier SE research 
conference, along with feeding its papers into the leading SE-related journals. 

• The SERC operates the world's largest and most-visited SE research website, 
including the largest and best-organized SE research experience base. It 
continues to provide leadership in evolving the SE Body of Knowledge. It runs 
the most widely-attended and highest-rated SE webcast series. 
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Implications: This vision requires significant investment in high-quality infrastructure, 
operations, and outreach. Such investment will require time to develop and mature. 

2 Sponsor Needs 


The DoD and the 1C have a continuing need for SE research. They face major challenges 
with respect to available SE methods, processes, and tools (MPTs), which have 
repeatedly been found wanting as described in more detail later in this section. 
Increasing system complexity, scale, and dynamism are driven by DoD's need to keep up 
with increasingly capable adversaries with widespread access to new technologies, 
along with challenges of legacy system interoperability and asymmetric warfare. In 
order to see-first, understand-first, act-first, and finish decisively, DoD needs its evolving 
systems to continue to interoperate readily and dependably. New technologies, such as 
smart systems, cloud services, powerful search engines, nanotechnology, and Internet- 
of-things, present new challenges to the SE discipline. New MPTs are needed in order to 
develop complex systems that are sustainable, affordable, and adaptive, while 
leveraging DoD's capital-intensive legacy. At the same time, these technologies may be 
commercially available to adversaries who can use them to improvise new threats. 

The National Research Council's "Human-System Integration in the System 
Development Process," (NRC, 2007), "Pre-Milestone A and Early-Phase SE," (NRC, 2008), 
and "Critical Code," (NRC, 2010) studies consistently found that the SE MPTs for 
integrating hardware engineering, human factors engineering, and software engineering 
into a scalable, unified approach were not up to the challenges of the complexity, scale, 
and dynamism characterizing DoD's large-scale systems and systems of systems. SE 
must enable rapid, affordable, trusted, and agile development of systems that are 
themselves rapidly fielded, affordable, trusted, and agile. The frequently ossified MPTs 
and inadequately prepared workforce common to today's acquisitions cannot meet 
those challenges. SE research is required to develop responsive MPTs and a fully 
qualified workforce. 

The 2009 Weapons System Acquisition Reform Act (WSARA, 2009) responded to the 
chronic shortfall in capabilities of the SE acquisition workforce. Among other steps, it 
demands innovative research to successfully address the increasing complexity and 
scale of both the systems DoD acquires and the dynamic and highly budget constrained 
environment in which those acquisitions occur. Related DoD directives such as DoD 
Instruction 5000.02, "Operation of the Defense Acquisition System," (DoD, 2008), the 
USD(AT&L) Ashton Carter memorandum, "Better Buying Power: Mandate for Restoring 
Affordability and Productivity in Defense Spending," (Carter, 2010), "Better Buying 
Power 2.0: Continuing the Pursuit for Greater Efficiency and Productivity in Defense 
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Spending" (Carter, 2012), and the Defense Business Board study "Linking and 
Streamlining the Defense Requirements, Acquisition, and Budget Processes," (DBB 
2012) also emphasize the need to more strongly integrate the SE function and the 
system cost and schedule analysis functions. This involves not only organizational and 
procedural improvements but also research into tradeoffs between system cost, 
schedule, and performance attributes; techniques for determining the ROI for 
investments in system flexibility and resilience; techniques for expediting the SE 
function; and improved capabilities for estimating and managing the cost and schedule 
of DoD software-intensive systems. 

Another highlighted critical area (DoD, 2008a; DoD, 2009; and DSB, 2007) is the need for 
more effective cyber-security of DoD information assets. This involves complementing 
the existing network and perimeter protections, which are primarily reactive, with more 
pro-active and system-aware protections. These include the ability to (a) combine 
techniques developed for the design of resilient systems with advanced cyber security 
techniques for achieving protection at the application layer of a system, including 
diverse redundancy and configuration hopping; (b) utilize system of systems design 
opportunities as a more resilient base for responding to cyber-attacks; and (c) combine 
techniques developed for automatic control systems in a manner that will both enable 
defense systems to determine if operator displays are being manipulated and serve as a 
basis for making rapid forensic analyses after a cyber-attack has possibly occurred. 

The 2010 DoD Science and Technology (S&T) priorities, presented by the Honorable 
Zachary J. Lemnios in his testimony to the House Committee on Armed Services on the 
FY13 budget request (Lemnios, 2012), respond to the complexity, scale, and dynamism 
of future DoD systems, which will continue to accelerate, making even stronger 
demands on DoD SE technology. Five of the seven DoD S&T priorities identified in his 
testimony — Engineered Resilient Systems, Cyber Science and Technology, Autonomy, 
Data to Decisions, Human Systems - are SE-intensive. All demand extensive SE research. 

3 SERC Response 


The SERC actively manages its research portfolio, looking for and nurturing synergies 
between projects. Additionally, the SERC works with its sponsor to identify 
opportunities where increased funding and project duration can produce higher positive 
impact on DoD's strategic SE research needs. This has been especially evident during 
2012 and 2013, in which a majority of new funding has extended existing projects rather 
than begin new ones. Most projects are now being conceived and proposed as multi¬ 
phase, multi-year efforts; for example, the Experience Accelerator Project, which is 
attempting to develop ways to greatly reduce the time needed to develop a mature 
systems engineer, is being executed as a 3-year project to deliver a strong foundational 


Approved October 25, 2013 


7 


SERC 2014-2018 Technic a I Plan 



Unlimited Distribution 


capability. Additional sponsors and funding are being sought to continue growing that 
capability and to deliver greater value after those first three years. 

In coordination with its sponsors, the SERC has structured its research portfolio into the 
following four thematic research areas, also depicted in Figure 3-1. 

• Enterprise Systems and Systems of Systems: The evolving need of very large 
scale systems composed of smaller systems, which may be technical, socio- 
technical, or even natural systems. These are complex systems in which the 
human behavioral aspects are often critical, boundaries are often fuzzy, 
interdependencies are dynamic, and emergent behavior is the norm. Research 
must enable prediction, conception, design, integration, verification, evolution, 
and management of such complex systems. 

• Trusted Systems: The need for ways to conceive, develop, deploy and sustain 
systems that are safe, secure, dependable and survivable. Research must enable 
prediction, conception, design, integration, verification, evolution and 
management of these emergent properties of the system as a whole, recognizing 
these are not just properties of the individual components and that it is essential 
that the human element be considered. 

• Systems Engineering and Systems Management Transformation: The need for 
ways to acquire complex systems with rapidly changing requirements and 
technology, which are being deployed into evolving legacy environments. 
Decision-making capabilities to manage these systems are critical in order to 
determine how and when to apply different strategies and approaches. Research 
must leverage the capabilities of computation, visualization, and communication 
so that systems engineering and management can respond quickly and agilely to 
the characteristics of these new systems and their acquisitions. 

• Human Capital Development: The need to respond to the retirement of the baby 
boomer generation, the reduced numbers of US citizens entering the technical 
workforce and the new systems challenges facing technical staff. Research must 
determine the critical knowledge and skills that the DoD and 1C workforce 
require as well as determine the best means to continually impart that 
knowledge and skills. 
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Figure 3-1. The four thematic areas being addressed by SERC research tasks and priorities 


These thematic areas have been further divided into eleven programs, shown below, 
that have the potential to make a transformative impact on DoD and the 1C. The SERC 
Research Council, which includes some of the most capable researchers in the field, 
helped shape this portfolio. 

• Enterprise Systems and Systems of Systems 

Enterprise Modeling: Model the socio-technical aspects of complex systems 
of system and enterprise systems, including developing and populating a 
framework to integrate models created using diverse methods and tools 

System of Systems Modeling and Analysis: Develop and test MPTs for 
analyzing and evolving systems of systems and provide support for their 
technical assessment in an Analytic Workbench construct 

• Trusted Systems 

Systemic Security: Create, validate, and transition MPTs to ensure systemic 
security using knowledge of system objectives and operation 

Systemic Assurance: Provide systemic assurance of safety, reliability, 
availability, maintainability, evolvability, and adaptability 
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• Systems Engineering and Systems Management Transformation 

- Affordability and Value in Systems: Create, validate, and transition MPTs to 
make better decisions on affordability and value in systems 

Quantitative Risk: Create, validate, and transition MPTs to improve risk 
identification, analysis tracking and management in acquisition and 
sustainment programs 

Interactive Model-Centric Systems Engineering (IMCSE): Create, validate, and 
transition MPTs to rapidly model the critical aspects of systems, especially 
those that facilitate collaborative system development 

- Agile Systems Engineering: Create, validate, and transition MPTs that enable 
rapid and flexible SE that can be applied for many kinds of systems in many 
types of development contexts 

• Human Capital Development 

Evolving Body of Knowledge: Establish active communities and mechanisms 
that create and support living bodies of SE knowledge 

Experience Acceleration: Develop an open source community that creates, 
validates, and transitions technology and content for the use of experiential 
technology to educate systems engineers and technical leaders 

SE and Technical Leadership Education: Create, validate, and transition 
curricula and practices to support the instruction and learning of systems and 
technical leadership for inexperienced students in college and experienced 
professionals 

Each of these eleven programs conducts research, to varying degrees, in the following 
synergistic areas: 

1. Principles, Methods, and Techniques: Develop improved ways to perform some 
aspect of SE and provide the underlying evidence for the expected results 

2. Analysis Tools: Research means to create better analysis tools; automate 
methods and techniques to enable their application by practitioners 

3. Simulation: Develop simulation technology to deepen system understanding and 
facilitate the effective application of SE MPTs 

4. Knowledge Transfer. Determine how practitioners, researchers, and students can 
more easily and rapidly translate SE research findings into practice 
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4 Objectives, Approach and Process 


4.1 Objectives 

The SERC will have the greatest impact on DoD and the 1C by: 

1. Conducting long-term research that makes significant progress on the grand 
challenges 

2. Transitioning that research into practice within DoD, the 1C, defense industrial 
base, and other federal agencies; and by developing more powerful ways to 
facilitate such transition 

3. Amplifying sponsor resources by forging relationships with other organizations 
that become partners, contributing their resources and energy to the SERC and 
adopting SERC research 

4. Strengthening the existing SERC culture, mechanisms and focus on collaboration 

5. Instituting new approaches to educate future systems engineers and engineers 
that leverage the full strength and diversity of the collaboration 

These objectives align with the SERC's four Operational Principles: 

1. Conduct innovative, high-impact research 

a. Only perform tasks which are research oriented (usually publishable 
when not classified) 

b. Focus research efforts on systems which can be generalized beyond a 
given domain and transform the discipline 

c. Focus on research efforts that have the potential of increasing the 
security and prosperity of the nation 

d. Focus on research which addresses future systems needs 

2. Translate proof-of-principle prototypes to impactful applications 

a. Work to ensure that there is a path from research results to impact 

3. Strengthen and leverage the research network 

a. Ensure that the research is conducted by the best available resources 

b. Bring in new Collaborators who provide long-term strategic benefit 

c. Focus on creating a network of academics, industry and government that 
is sustainable 

4. Prepare the next generation 

a. Provide a focus on education and training research, both in research 
(graduate students) and practitioners 
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4.2 Approach 

The general approach to creating the SERC 2014-2018 Technical Plan was first to identify 
Grand Challenge problems in each of the four SERC thematic research areas shown in 
Figure 3-1. These Grand Challenges were formulated to provide a point of integration 
between existing programs in each research focus area, and also provide opportunities 
to generate new, related research areas. These Grand Challenges also provide 
inspiration and an integration point for non-SERC universities, federally funded research 
and development centers (FFRDCs), and industrial researchers to perform collaborative 
research and provide natural transition into use. 

Additionally, the Plan assumes: 

1. Incentives will be established so that future funding is dependent on principal 
investigators (Pis) and researchers finding funding for their programs outside the 
sponsorship. This could come in the form of matching funds or other incentives. 

2. Incentives will be established for Pis to transition their research into practice. 
Each project will have a transition plan in place when the project begins with the 
opportunity for additional downstream funding to facilitate transition to practice 
and to develop educational materials and courses based on research results that 
will be shared by all SERC collaborating institutions. 

3. Seed funding will be available to explore novel and promising ideas that may be 
the sources of future breakthroughs. These ideas will be selected by the 
sponsors, SERC Research Council and SERC leadership through an open 
solicitation process with all of the SERC Collaborators. 

4. Shared IT infrastructure will be available for use by every research program. 


4.3 Process 

SERC management and the SERC sponsor set the overall objectives and framework for 
this Plan. SERC management then worked with the Research Council, Program Leads 
and others to craft the Grand Challenges, objectives, and strategy for each of the four 
research focus areas, and to lay out program descriptions, timelines, anticipated results, 
and resources required. 

5 Focus Areas, Programs, and Projects 


Programs in each of the four research focus areas and in supporting activities are 
described in the five subsections below. 
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5.1 Enterprise and Systems of Systems (ESOS) 

Each DoD/IC Service and Agency, and the DoD itself, is an example of "enterprises as 
systems". Such organizations have the challenge of integrating and evolving multiple 
portfolios of systems with often-conflicting sets of objectives, constraints, stakeholders, 
and demands for resources. Systems of systems generally involve integrating multiple 
independently managed systems to achieve a unique capability, therefore involving 
needs for negotiation as well as control. Thus, in some contexts, systems of systems are 
enterprise challenges as well. Indeed, both "enterprises as systems" and systems of 
systems increasingly face situations in which the classical systems approach of 
deterministically engineering the system based on relatively static requirements and 
specified human interactions are insufficient. In such complex systems, human 
behavioral and social phenomena are critical as are cascading impacts from 
interdependencies; altogether emergent outcomes are the norm. Research is necessary 
to determine the foundational SE principles for such systems. Those principles can then 
be used to develop associated SE MPTs applicable to such complex systems. 

Goal: Improved engineering to develop and deliver end-to-end defense capability to the 
warfighter for operation in complex organizational and operational environments, with 
fewer unintended negative consequences and greater resilience 


5.1.1 ESOS Grand Challenge 

The ESOS grand challenge to achieve the ESOS goal is to: 

Create the foundational SE principles and develop the associated MPTs to enable 
the DoD and its partners to model (architect, design, analyze), acquire, evolve 
(operate, maintain, monitor) and verify complex enterprises and systems of 
systems to provide the DoD with an affordable and overwhelming competitive 
advantage over its current and future adversaries. 


5.1.2 Strategies to Address the ESOS Grand Challenge 

Successfully executing the following strategies will make significant progress towards 
addressing the ESOS Grand Challenge: 

1. Model: Develop MPTs that allow quick and insightful modeling of 

enterprises/SoSs so that the effects of changes in policies, practices, 
components, interfaces, and technologies can be anticipated and understood in 
advance of their implementation 
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2. Acquire: Develop MPTs that allow insight into enterprise/SoS acquisition 
approaches in the face of significant uncertainty and change to minimize 
unintended consequences and unforeseen risks 

3. Evolve: Develop MPTs that facilitate evolving and growing an enterprise/SoS, 
including insight into different architectural and integration approaches that 
facilitate evolution in the face of uncertainty and change in how an 
enterprise/SoS is employed, the technologies available to realize it, and the 
environment in which it exists 

4. Verify: Develop MPTs that allow the properties of an enterprise/SoS to be 
anticipated, monitored and confirmed during development and evolution, 
including an enterprise/SoS which includes legacy systems that are in operation 
while development and evolution are underway 

Two research programs, described below, directly implement the strategy. Enterprise SE 
and Modeling and Systems of Systems (SoS) Analysis. Possible future projects were 
defined at the ESOS Summit held in on May 7-8 th 2013 in Washington DC and are 
discussed after the ongoing projects. 


5.1.3 Enterprise SE and Modeling Program 

2012 and 2013 have been focused on developing a rigorous systems science and 
engineering foundation for ESOS, and building a community of researchers who 
collectively will advance ESOS research. In addition, ESOS researchers have been 
developing domain-specific multi-level or multi-scale models in areas such as 
counterfeit parts, healthcare delivery, and urban resilience with support from a variety 
of sponsors. 

This foundation and experiences have provided the basis to take on the broader goal of 
providing enterprise level process mapping, monitoring and control for the grand 
challenge of Enterprise and Systems of Systems. As shown in Figure 5.1-1, data, 
forecasts and reports (including text) flow from the context of an enterprise's ongoing 
transformation; data and text mining are used to make sense of this flow of information; 
the insights gained enable computational modeling; the results of which are used to 
drive interactive visualizations that enable process mapping, monitoring and control. 

Pursuit of this grand challenge represents a transition from focusing solely on the design 
of enterprises and systems of systems to the design and operation of enterprises and 
systems of systems. In other words, the computational models and interactive 
visualizations would be run in parallel with actual operations and provide a means for 
monitoring and the control of operations. This would enable the operators of complex 
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systems to detect, diagnose, and compensate for deviations of operations from 
expectations. The complex enterprise systems of interest could range from military 
operations and acquisition programs, to urban infrastructures and healthcare delivery 
organizations. 



Interactive 

Visualization 

Computational 

Modeling 

Data & Text 
Mining 

Context of 
Transformation 



Figure 5.1-1. Enterprise Process Mapping, Monitoring and Control Communication Flows 

This research program implements ESOS strategies 1, 2 and 3, currently, and will be extended 
to 4 as well. It includes one project already underway: Multi-Level Modeling of Socio-Technical 
Systems. 

Table 5.1-1 offers a description of this project and which strategy it primarily supports. 

Table 5.1-1. Enterprise SE and Modeling Projects 


Projects 

Started Purpose 

Primary ESOS 
Supported 
Strategies 

Multi-Level Modeling of 
Socio-Technical Systems 

Develop an integrating framework to support multi¬ 
level modeling of socio-technical SoS systems 

1 


5.1.3.1 Multi-bevel Modeling of Socio-lechnical Systems Project 

Over several decades, the Department of Defense has invested $100B developing 
computational models and simulations of complex military systems. These estimated 8,000 
software artifacts include 100 million lines of code and model everything from weapon 
platforms to operational military organizations. Unfortunately, they rarely interact with each 
other. Existing capabilities must be enhanced to allow models of different levels of fidelity to 
interoperate in a dynamic fashion. 
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This research shall perform an architectural assessment to determine the technical challenges, 
options and recommended architectural approaches involved in developing a modeling and 
simulation (M&S) framework to support the desired capabilities. The goal of the research is to 
review existing data structures, models, approaches, and technology to define an architecture 
that is as close to a complete solution to the problem as possible. The SERC will develop 
scenarios to test the completeness of the proposed architectures. These scenarios will, at a 
minimum, explore the impacts of the envisioned M&S framework on technology, military 
methods and organizational interactions. The SERC will explicitly identify critical technological, 
organizational, and other challenges to the development or use of the framework. The SERC 
will also provide an unambiguous statement regarding the common limitations of the 
elemental models and their aggregations within the framework. Future research will attempt 
to develop an integrating framework. It will formulate a multi-level model platform, (including 
alternative representations and approaches) and design a human-computer interface with rich 
visualizations and scenario building controls. 

Table 5.1-2 shows the focus, deliverables, and investment in Multi-Level Modeling of Socio- 
Technical Systems through 2018. 


Table 5.1-2. Multi-Level Modeling of Socio-Technical Systems Project Timeline 


Year 

Focus 

Key Deliverables 

Pre-2014 

Architectural Assessment of existing 
commercial practices 

Technical report, summary of interviews and 
observations 

2014-15 

Develop Integrating Framework 

Technical Report 

2016-18 

Develop Modeling Prototype 

Demonstration of prototype, code base, technical 
reports 


5.1.4 Systems of Systems (SoS) Analysis Program 

Developing a system of systems remains a highly challenging endeavor. The complex 
interdependencies among systems often exhibit managerial and operational independence, yet 
must work cohesively to achieve an overarching set of capabilities. Trades between capability 
and risk are essential decisions that must be addressed for SoS capability planning. Existing 
tools for such trades are of limited value when size and/or interdependency complexity is high. 

This research program addresses the need to create and mature decision-support tools 
specifically for evolving SoS architectures; in particular, those technical architectures that 
support assessing the impact of potential disruptions during development or operation. The 
initial research explores analytical methods to quantify the impact of system interdependencies 
in the context of SoS capability development as well as broader agent models that address the 
often fuzzy influence of stakeholder perspectives in the technical development activities. 
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Additional research will focus on identifying innovative approaches to support SE in 
architecting, engineering, and evolving complex SoS. 

This research program implements ESOS strategies 1, 2 and 3, currently, and will be extended 
to 4 as well. It includes one project already underway: Systems of Systems (SOS) Analysis. Table 
5.1-3 offers a description of this project and which strategies it primarily supports. 

Table 5.1-3. System of Systems Analysis Projects 


Project 


Systems of Systems (SOS) 
Analysis 


Started Purpose 


Develop MPTs and an Analytic Workbench construct to 
2011 house them for the purpose of SoS architecture 
evaluation, selection and evolution. 


Primary ESOS 
Supported 
Strategies 


1, 2,3 


5.1A1 MPTs fora Systems of Systems Analytic Workbench Project 

The objective is to develop a framework for an Analytic Workbench that presents practitioners 
with appropriate methods and tools that can characterize complex SoSs, dynamically analyze 
and assess SoS performance under changing conditions and scenarios, develop SoS 
architectures that effectively address the independent nature of constituent systems and their 
unanticipated change, and improve user capabilities given SoS complexities and risks. 

Table 5.1-4 shows the focus, deliverables, and investment in Systems of Systems (SoS) Analysis 
through 2018. 


Table 5.1-4. MPTs for a SoS Analytic Workbench Project Timeline 


Year 

Focus 

Key Deliverables 

Pre-2014 

Evaluate architectures and Develop 
Proof of Concept simulation 

Technical reports, agent based model for selected 
application domain 

2014-15 

Develop prototype tools 

Technical reports, demonstration of prototype 

2016-18 

Refine and extend prototype tools 

Technical reports, demonstration of prototype 
application 


5.2 Trusted Systems (TS) 

The organization of its assets into net-centric systems of systems (NCSOS) has enabled DoD to 
much more rapidly and effectively see-first, understand-first, act-first, and finish decisively in its 
operations. However, this implies that each of its assets needs to achieve higher levels of trust 
as part of the NCSOS, as compared to its previous role as a standalone platform, while retaining 
or improving its previous speed and effectiveness. 
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The SERC Trusted Systems research area addresses this challenge by balancing traditional 
reactive cyber-security defenses with pro-active mechanisms, making attacks more risky and 
expensive. It combines this approach with increased and continuous application of advanced 
assurance capabilities that concurrently address not only security but also safety, reliability, 
availability, maintainability, usability, interoperability, and resilience. 

Goal: Transform system assurance from a late, reactive activity to an early and continuous, 
pro-active orchestration of advanced assurance MPTs, in ways that balance the simultaneous 
achievement of cyber-security trust and assurance with complementary MPTs for assuring safe, 
reliable, available, usable, interoperable, and resilient mission cost-effectiveness. 


5.2.1 TS Grand Challenge 

The TS grand challenge to achieve the TS goal is to: 

Achieve much higher levels of system trust by applying the systems approach to 
achieving system assurance and trust for the increasingly complex, dynamic, cyber¬ 
physical-human net-centric systems and systems of systems of the future. 


5.2.2 Strategies to Address the Trusted System Grand Challenge 

Successfully executing the following strategies will make significant progress towards 
addressing the TS Grand Challenge: 

1. Design for System Assurance and Trust: Develop design patterns and systems 
architectures, with corresponding systems engineering principles guiding application, 
and associated design analysis MPTs for early assurance of needed properties 

2. Understand the Cost of Assurance and Ensure Cost-Effective Assurance: Develop MPTs 
that enable understanding, predicting, and ensuring the cost-effectiveness of 
implementing high-assurance policies and requirements, especially on complex systems 
and complex systems of systems 

3. Understand and Ensure Balanced Tradeoffs Between Assurance "ilities" and Other 
"llities": Develop MPTs that enable understanding, predicting, and ensuring cost- 
effective relationships among assurance policies/requirements and other "ilities", such 
as usability, interoperability, and maintainability 

4. Measure System Assurance: Develop MPTs that allow measuring "how much" 
assurance of needed properties a system has, and that permit comparison of the 
relative assurance and trust provided by alternative systems 
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Two SERC research programs directly implement the strategies: 

• Systemic Security. The most compelling need for assurance of trust is in the area of 
system security. Given the numerous sources of security breaches available at low cost 
to attackers, a major concern is to make DoD systems, systems of systems, and 
enterprises harder and harder to attack, while simultaneously making them more risky 
and expensive to penetrate and damage. 

• Systemic Assurance. Besides security attacks, there are numerous sources of system 
disruption such as natural disasters, system misuse, system overload, system 
component wear out, and defects in a system's requirements, design, or construction 
that cause loss of stakeholders' lives, health, capability, property, or financial assets that 
need significant improvements in trust not only for current systems, but for the more 
complex and dynamic DoD systems, systems of systems, and enterprises of the future. 

In addition, improvements in system trust have been and are being addressed in the other 
research areas, particularly in SE and Management Transformation and its current projects in 
ilities Tradespace and Affordability, Interactive Model-Centric SE, and Quantitative Risk. 
Example contributions from these and earlier SEMT projects include SERC insights such as those 
from the RTs addressing technical, integration and manufacturing maturity level assessment, 
the 35 Risk Management Precepts, the RT-EM approach to quantifying early-SE risks, the MIT 
epoch-era approach to assurance under uncertainty, and the set-based vs. point-design 
approach to assurance of systems undergoing continuing and extensive change. The synergies 
among these research projects will be addressed and enhanced by periodic cross-research-area 
workshops. 


5.2.3 Systemic Security Program 

Goal: Develop safe, secure, dependable defense systems that are resilient to cyber and other 
threats through systemic security approaches that complement current, incomplete 
perimeter/network. 

Approach: Reversing cyber security asymmetry from favoring our adversaries (small investment 
in straightforward cyber exploits upsetting major system capabilities), to favoring the US (small 
investments for protecting the most critical system functions using System Aware cyber security 
solutions that require very complex and high cost exploits to defeat). 

Strategy: 

1. Design for System Security: Develop design patterns and security architectures, with 
corresponding systems engineering principles guiding application, that enable security 
to be based on the specific properties of the system and its implementation rather than 
on traditional perimeter strategies 
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2. Understand the Cost of Security and Ensure Cost-Effective Security: Develop MPTs that 
enable understanding, predicting, and ensuring the cost-effectiveness of implementing 
specific security policies and requirements, especially on complex systems and complex 
systems of systems 

3. Understand and Ensure Balanced Tradeoffs Between Security "ilities" and Other "llities": 
Develop MPTs that enable understanding, predicting, and ensuring cost-effective 
relationships between specific security policies/requirements and other "ilities", such as 
reliability, safety, and maintainability 

4. Measure System Security: Develop MPTs that allow measuring "how much" security a 
system has and that permit comparison of the relative security between two alternative 
systems 

This research program implements all four TS strategies above. It includes one project already 
underway which two Research Topics have supported. Table 5.2-1 offers a description of this 
project and which strategies it primarily supports. 

Table 5.2-1. Systemic Security Projects 




Primary TS 

Projects 

Started Purpose 

Supported 



Strategies 

Piloting and Extensions of 
Systems-Aware Security 

2011 Re f' ne systems-aware MPTs developed earlier and pilot 

them in multiple application areas 

1, 2, 3, 4 


5.2.3.1 Piloting and Extensions of Systems-A ware Security Project 

In 2011, SERC RT-28: Systems-Aware Security developed a rapid prototype security capability 
that includes 1) data continuity checking within the application, 2) real-time virtual 
configuration hopping of selected C2 functions across multiple operating systems to provide 
defense through diversity, 3) real-time physical configuration hopping to both provide defense 
through diversity and resilience in the face of successful attacks, and 4) a closed loop control 
system for automatic restoration from a successful attack. 

In 2012-2013, SERC RT-42: Security Engineering Pilot developed a prototype flight-capable 
security capability directed toward an unmanned air vehicle (outlaw aircraft containing an 
embedded Cloud Cap Piccolo flight control system) carrying an already selected set of 
surveillance equipment (video/IR cameras, radar, and a SIGINT package). 

The continuation of this project will initially refine the Systems Engineering Pilot capability 
based on the preliminary evaluations. It will then conduct more thorough and realistic 
evaluations, further refine the capabilities, and package them to be tailorable to other domains. 
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Subsequent activities will involve the tailoring of the capabilities to other domains, and 
associated evaluation and refinement, along with monitoring and refinement of existing fielded 
capabilities. 

Table 5.2-2 shows the focus, deliverables, and investment in the project through 2016. 
Continuing extensions and upgrades will be pursued in 2017 and 2018. 


Table 5.2-2. Piloting and Extensions of Systems-Aware Security Project Timeline 


Year 

Focus 

Key Deliverables 

Pre-2014 

Concept definition, prototyping, piloting 

Initial Systems-Aware Security capability 

2014 

Extended evaluation, refinement, and 
Packaging 

Tailorable Systems-Aware Security capability 

2015 

Initial new-domain tailoring, evaluation, 
and refinement 

New-domain Systems-Aware Security capability 

2016 

Two new-domain tailoring, evaluation, 
and refinements 

Two New-domain Systems-Aware Security capabilities 


5.2.4 Systemic Assurance Program 

Besides security, the engineering of resilient DoD systems requires assurance of safety, 
reliability, availability, durability, survivability, maintainability, evolvability, adaptability, and 
sustainability. All of this assurance needs to be achieved for increasingly complex, dynamic, 
cyber-physical-human net-centric systems, systems of systems and enterprises with needs for 
rapid response incompatible with most heavyweight assurance MPTs. Carnegie Mellon 
University has been a leader in developing assurance MPTs in such areas as model checking, 
architectural style analysis, race and deadlock detection for multicore and other concurrent 
systems, appropriate test case generation, and assurance-case analysis that are not only more 
powerful in anomaly detection, but also leading to stronger possibilities for positive assurance 
and to greater scalability to large systems and to more rapid execution. These have been 
successfully applied in such areas as the High Level Architecture analysis for networked DoD 
models and simulations, cyber-physical robotic systems, and extremely large commercial Java 
programs. 

The Systemic Assurance project builds on these capabilities and data sources to address their 
integration to develop incrementally composable combinations of MPTs and data-based 
composition guidance for obtaining the most cost and schedule-effective combinations for the 
assurance of necessary system properties. Particular areas of emphasis will be exploiting 
analogies with successful techniques in other domains, such as building codes; better 
management of chains of evidence to support ongoing re-evaluation for rapidly evolving 
systems both in development and in sustainment/modernization; enhancing MPTs with 
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language extensions for assurance assertions or context metadata; support for dynamic 
adaptiveness and resiliency in architectural design; and data management and metrics for more 
evidence-based design, development, and decision support. 

Reflecting strong natural ties between the Systemic Assurance Project, the Piloting and 
Extensions of Systems-Aware Security Project, and the ilities Tradespace and Affordability 
Project (iTAP) described in 5.3.3.1, extensive coordination will be pursued among the three 
projects. 

This research program implements all four Trusted Systems strategies above. It includes seven 
research and technology subprojects led by senior CMU researchers, with extensive internal 
coordination mechanisms to exploit the synergies among the various technical approaches. 
The CMU team members have an extensive network of collaborations and partnerships with 
DoD activities, other government agencies, the Software Engineering Institute, other SERC 
universities, and numerous industrial firms. The team will build on these to acquire access to 
data and technical partners. The seven subprojects are: 

1. Develop baseline and intervention models for a selection of current standards and 
practices (identified in collaboration with DoD stakeholders), refining technical 
understanding of gaps and limitations. This baselining effort is essential to support a 
measurement-based approach to documenting the impact of the proposed new 
technologies and process interventions. This includes identifying the key criteria and 
dimensions of measurement. 

2. Undertake engineering design effort focused on integrating improved capability for 
traceability and other features required to support explicit modeling and management 
of chains of evidence. A key focus is to demonstrate that it is possible to enhance 
existing tools and environments, including both integrated development environments 
(IDEs) and team tools, with relatively little disruption to established team practices and 
metrics. 

3. Design and implement experiments to address the challenge of rapid recertification. 
These include capturing evidence and assurance-related reasoning (assurance cases, 
models, analyses, configuration management, etc.). This area of rapid recertification is 
critical to iterative, incremental, and staged development practices. It is also critical to 
systems with supply chains that include externally developed components and 
infrastructure such as commercial and open-sourced databases, operating systems, 
frameworks, and libraries. (Almost all larger-scale software-reliant systems have this 
characteristic.) 

4. Develop a framework for assessment of architecture-derived quality attributes, focusing 
on architectural modeling and the relationship of architectural and compositional 
models with quality outcomes. This is essential in order to ensure that key decisions 
made at early lifecycle phases will have intended quality outcomes. 
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5. Develop requirements elicitation and management approaches that better address 
quality and policy objectives. Requirements elicitation and management is one of the 
earliest areas of focus in an engineering process, and decisions at this point can have 
tremendous leverage on quality outcomes. This work is directed at providing more 
immediate assessments of the potential outcomes of early requirements-related 
decisions. By improving models, it becomes possible to better manage the linkage of 
requirements and architectural decisions. 

6. Augment and collaborate with diverse existing efforts focused on technical means to 
address particular quality criteria. Many of these quality criteria are emerging as 
significant challenges because they tend to defy conventional testing and inspection 
techniques. These include, for example, a number of attributes related to safe 
concurrency, compliance with application program interface rules-of-the-road, cyber¬ 
physical architectural compliance, state and access management for shared objects, 
taint and flow and other security-related attributes, and others. 

7. Identify and advance areas in support of increasing automation, in order to reduce 
workload of developers and evaluators and to advance existing workload forward in the 
process, with immediate rewards. The purpose of this is to frame an ultimately more 
quantitative business case for adoption based on increased return on investment for 
assurance-related effort and reduced uncertainty (lesser variance in estimate "cones"). 
This is supportive of the longer-term goal of a "positive benefit" model for the adoption 
of assurance-related practices. It also supports a stakeholder-engaged process model 
analogous to building codes. 

There are important synergies and interactions among these seven subprojects, with the 
principal features outlined in Figure 5.2-1 below (subproject numbers are in brackets). 
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System architectural attributes 
(resiliency, composition, cyber¬ 
physical, quality attributes) [4] 


Requirements elicitation and 
modeling 

(formal traceability, analysis) [5] 


7 


Technical attribute 
modeling, evaluation, and 
assurance [6] 


Team practices and data- 
intensive tools 
(chains of evidence for 
continuous quality-focused 
evidence production H21 


Quality-attribute gap analysis 
(DoD baseline practices) [1] 



Rapid/continuous evaluation of 
evolving systems 
(devt and experimentation) [3] 


Systemic Assurance 
Principal Task Interdependencies 


Enhanced tool-supported 
practices, automation, 
metrics, business case [7] 


Figure 5.2-1. Systemic Assurance Principal Task Interdependencies 


Table 5.2-3 below offers a description of this project timeline. 

Table 5.2-3. Systemic Assurance Projects 


Projects 

Started Purpose 

Primary TS 
Supported 
Strategies 

Systemic 

Assurance 

Inputs from previous and ongoing SERC areas, programs and projects, 
2013 including Systemic Security, Systems Engineering and Management 

Transformation, and Systems of Systems and Enterprises programs 

1, 2, 3, 4 


5.2A1 Systemic Assurance 

Table 5.2-4 shows the focus, deliverables, and investment in Systemic Assurance through 2016. 
Continuing extensions and upgrades will be pursued in 2017 and 2018. 


Approved October 25, 2013 


24 


SERC 2014-2018 Technic a I Plan 







Unlimited Distribution 


Table 5.2-4. Systemic Assurance Project Timeline 


Year 

Focus 

Key Deliverables 

Pre- 

2014 

Startup 

Prepare for and hold kickoff meeting with sponsor. 
Systemic Security, and iTAP representatives 

2014 

Initiate efforts in all seven thrusts. 

Identify principal candidate software code bases and 
cyber-physical systems for empirical studies. Candidates 
include commercial and government partner systems 
(depending on access) as well as significant open source 
systems of government interest such as Hadoop, SE 

Linux, Android SE, and others. Begin empirical studies 
and solution explorations. 

2015 

Identify most promising solution options, 
begin solution research and development 

In thrust 1, complete baseline analysis and identify 
feature points to address in framing potential revised 
practices. Identify technical blockers and potential 
remediations. 

Initiate work on tool design, building on capabilities of 
established IDE and team tools. Develop a "minimal 
perturbation" model that developers will find familiar 
but that augments the tooling in specific ways to 
address the traceability, modeling, and analysis 
challenges of thrusts 2, 3, and 7. 

Continue work on specific technical attributes, focusing 
on the challenges of attributes that tend to defy 
conventional testing and inspection (thrusts 4 and 6). 

Develop assessment techniques to address the 
requirements elicitation goals of thrust 5. 

2016 

Elaborate and mature solutions; engage 
with stakeholders; conduct trial 
deployments. Identify and explore new 
areas of high-potential research 

Engage with stakeholders (potentially in collaboration 
with SEI) to continue the baselining and criteria 
definition of thrust 1, leading to a preliminary 
formulation of an alternative model based on building 
code ideas (thrusts 2, 3, and 7). 

Advance efforts in technical and tooling thrusts, 
producing exemplar evidence-based assurance data for 
existing major components. 

Conduct trial deployments of advanced tooling and 
metrics capabilities with professional development 
teams in partner organizations. 

Advance requirements and architecture efforts, 
identifying candidate "emerging best practices" to 
support architecture-led iterative development efforts. 
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Year 

Focus 

Key Deliverables 



Demonstrate rapid recertification for one or more of 
the exemplar systems for which evidence is created in 
year 3. 


Demonstrate and interact with baseline 

Develop evidence-based approaches for dynamic and 

2017 

capability users; extend and apply 

resilient systems potentially with shape-shifting 

additional solutions. Mature new areas of 

architectures. 


high-potential research 

Engage with stakeholders to advance experimental 
concepts for new evidence-based approaches to 
"designed-in assurance support" for larger-scale 
component-based systems. 


Based on experience with existing results. 

Advance the traceability capabilities in tooling to 
support a concept of continuous re-evaluation and 
reconstruction of evidence to support a model of 
"continuous re-certification." 

2018 

identify and pursue further baseline 
extensions and new-idea projects 

Enhance tool prototypes to include broader ranges of 
critical technical quality attributes (thrust 6). 



Identify advances in modeling, language, and analysis to 
enable broad adoption of evidence-based approaches. 


5.3 Systems Engineering and Systems Management Transformation (SEMT) 

Traditional DoD systems engineering and management (SE&M) practices have been focused on 
the definition and acquisition of individual standalone platforms within a relatively stable 
environment. They have generally used slow, sequential processes that often commit to 
requirements before their development implications are fully understood. This has caused 
much late and expensive rework, along with brittle, hard-to-modify architectures. 

However, DoD's current and future environment requires that such platforms and their human 
and software elements be defined, integrated, and evolved within highly complex and dynamic 
net-centric systems of systems and enterprises. This requires research focused on transforming 
traditional SE&M MPTs to meet these current and future DoD mission needs. 

Goal: Transform systems engineering and its associated management approaches away from 
systems designed for optimal performance against a static, pre-specified set of requirements 
over long procurement cycles to approaches that enhance the productivity of engineers to 
rapidly and concurrently develop cost effective, flexible, agile systems that can respond to 
evolving threats and mission needs. "Systems" covers the full range of DoD systems of interest 
from components such as sensors and effectors to DoD-wide net-centric systems of systems 
and enterprises. "Effectiveness" covers the full range of needed system quality attributes or 
ilities, such as reliability, availability, maintainability, safety, security, performance, usability. 
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scalability, interoperability, speed, versatility, flexibility, and adaptability, along with composite 
attributes such as resilience, suitability, and sustainability to support the desired mission 
performance. "Cost" covers the full range of needed resources, including present and future 
dollars, calendar time, critical skills, and critical material resources. 


5.3.1 SEMT Grand Challenge 

The SEMT grand challenge to achieve the SEMT goal is to: 

Move the DoD community's current systems engineering and management MPTs and 
practices away from sequential, single stovepipe system, hardware-first, outside-in, 
document-driven, point-solution, acquisition-oriented approaches; toward concurrent, 
portfolio and enterprise-oriented, hardware-software-human engineered, balanced 
outside-in and inside-out, model-driven, set-based, full life cycle approaches. These will 
enable much more rapid, concurrent, flexible, scalable definition and analysis of the 
increasingly complex, dynamic, multi-stakeholder, cyber-physical-human DoD systems, 
systems of systems, portfolios of systems, and enterprises of the future. 


5.3.2 Strategy to Address the SEMT Grand Challenge 

Successfully executing the following strategies will make significant progress towards 
addressing the SEMT Grand Challenge: 

1. Make Smart Trades Quickly: Develop MPTs to enable stakeholders to be able to 
understand and visualize the tradespace and make smart decisions quickly that take into 
account how the many characteristics and functions of systems impact each other 

2. Rapidly Conceive of Systems: Develop MPTs that allow multi-discipline stakeholders to 
quickly develop alternative system concepts and evaluate them for their effectiveness 
and practicality 

3. Balance Agility and Assurance: Develop SE MPTs that work with high assurance in the 
face of high uncertainty and rapid change in mission, requirements, technology, and 
other factors to allow a system to be rapidly acquired and responsive to both 
anticipated and unanticipated changes in the field 

4. Align with Engineered Resilient Systems: Align research to both leverage the research 
and technology results of the Engineered Resilient Systems (ERS) program, and 
contribute to it; e.g., ERS efforts to define new approaches to tradespace. 
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Four SERC research programs directly implement the strategies: 

• Affordability and Value in Systems 

• Quantitative Risk 

• Interactive Model-Centric Systems Engineering (IMCSE) 

• Agile Systems Engineering 


5.3.3 Affordability and Value in Systems Program 

The Affordability and Value in Systems Program will use the full range of DoD stakeholder value 
propositions to structure and pursue a value-based ilities hierarchy. Its initial form includes the 
stakeholder values of having current-system Mission Effectiveness (Speed, Delivery Capability, 
Accuracy, Usability, Scalability, Versatility); current-system Resource Utilization Efficiency (Cost, 
Duration, Personnel, Scarce Quantities (size, weight, energy, ...), Producibility, Supportability); 
current-system Protection (Safety, Security, Privacy); current-system Robustness: (Reliability, 
Availability, Maintainability); along with future-system Flexibility (Modifiability, 
Tailorability/Extendibility, Adaptability); and future-system Composability 
(Interoperability/Portability, Openness/Standards Compliance, Service-Orientation) to address 
evolving field needs. 

This program will pursue the Grand Challenge of performing ilities tradespace and affordability 
analysis for cyber-physical-human systems and systems of systems in a portfolio and enterprise 
context. It will integrate current strengths in physical-systems tradespace analysis and 
information-systems tradespace analysis. It will pursue both basic research on the foundational 
relationships among ilities, and applied research on the pilot application and evolution of ilities 
tradespace and affordability MPTs within key DoD application domains, including full-coverage 
cyber-physical-human total ownership cost estimation models addressing the new 
characteristics of future DoD systems, systems of systems, portfolios, and enterprises. 

This research program primarily implements SEMT strategy one above. Table 5.3-1 offers a 
description of the ilities Tradespace and Affordability Project (iTAP), the one current project in 
the Affordability and Value in Systems Program. 

Table 5.3-1. Affordability and Value in Systems Projects 




Primary SEMT 

Project 

Started Purpose 

Supported 

Strategies 

iTAP 

2012 Pursue the Grand Challenge of performing ilities tradespace and 

affordability analysis for cyber-physical-human systems 

1 
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5.3.3.1 DAP Project 

Table 5.3-2 shows the focus, deliverables, and investment in iTAP through 2016. The timeline 
beyond 2016 has not yet been established. The iTAP program has two primary initial foci, and a 
third educational focus once a critical mass of iTAP capabilities have been developed. 

The first initial focus is on researching and developing the foundations of ilities Tradespace and 
Affordability (IT&A) analysis via a framework of IT&A views that aid in organizing and applying 
IT&A analysis to address the systems engineering of cyber-physical-human systems, systems of 
systems, portfolios, and enterprises. The views include DoD stakeholder value-based ility 
definitions, relationships, and priorities; means-ends views for achieving individual ilities; 
architectural strategies for achieving individual ilities and their second-order impacts on other 
ilities; process strategies for incrementally addressing uncertainties in ility tradespace 
situations, and for concurrently balancing a cyber-physical-human system's ility aspects; 
domain-specific ility tradespace views; and system of systems and enterprise views, including 
challenges in scalability and in reconciling the incompatible assumptions of component-system 
domain-specific architectures. 

The second initial focus is on extending and integrating existing IT&A MPTs to better support 
DoD cyber-physical-human system ility analysis. This will include developing more service- 
oriented and interoperable versions of current SERC ility MPTs; developing approaches for 
better integrating MPTs primarily focused on physical, cyber, or human system IT&A analysis; 
efforts to modify and compose existing SERC ility IT&A MPTs to better interoperate with each 
other and with counterpart MPTs in the ERS community and elsewhere; and efforts to apply the 
MPTs to the IT&A analysis of increasingly challenging DoD systems. In the affordability area, a 
particularly promising prospect is a collaborative SERC-Aerospace Corporation-USAF/SMC-NRO 
effort to develop an integrated lifecycle cyber-physical-human system cost model for satellite 
systems, including the flight, ground, and launch systems, which could be subsequently 
extended to other DoD domains, along with cost estimation at the portfolio and enterprise 
levels. 


Table 5.3-2. iTAP Project Timeline 


Pre-2014 


Research and develop basic iTAP 
concepts and framework. Explore early 
MPT applications and interoperability, 
including with ERS counterparts. 


Key Deliverables 


Basic iTAP concepts and framework. Results of early 
MPT applications and interoperability improvements. 
Prototype integrated lifecycle cyber-physical-human 
system cost model. Multi-view IT&A analysis guidance 
papers. 


Elaborate iTAP concepts and framework 
in key areas e.g., systems of systems. 

2014 MPT extensions; broader and deeper 
applications and interoperability. iTAP 
new-idea explorations. 


Elaborated iTAP concepts and framework. Results of 
broader and deeper MPT applications. Integrated 
lifecycle cyber-physical-human system cost model 
domain-specific IOC. Multi-view IT&A Analysis 
Guidebook v 0.5; associated papers. 
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2015 


Mature iTAP concepts, framework, 
Guidebook. Increasingly scalable and 
interoperable MPTs. Extensions of iTAP 
new-idea explorations. Guidebook- 
based outreach and educational 
initiatives 


Key Deliverables 


Mature iTAP concepts and framework. Results of 
increasingly scalable and interoperable MPT 
applications. Extended domain-specific lifecycle cyber- 
physical-human system cost model; prototypes in other 
domains, and for portfolio and enterprise extensions. 
Multi-view IT&A Analysis Guidebook v 1.0; Guidebook- 
oriented courseware, early usage at AFIT, NPS, DAU, 
SERC universities. 


2016 


Integration of new-idea explorations 
into iTAP concepts, framework, 
Guidebook. Increasingly scalable and 
interoperable MPTs. Further iTAP new- 
idea explorations. Guidebook-based 
outreach and educational initiatives 


New-idea-extended iTAP concepts and framework. 
Results of increasingly scalable and interoperable MPT 
applications. Extended multi-domain lifecycle cyber- 
physical-human system cost model, including portfolio 
and enterprise extensions; Multi-view IT&A Analysis 
Guidebook v 1.1; Guidebook-oriented courseware, 
broad usage at AFIT, NPS, DAU, SERC, and other 
universities. 


5.3.4 Quantitative Risk Program 

The Quantitative Risk Program, which currently has one project, primarily implements SEMT 
strategy 1 above. Table 5.3-3 describes the Quantitative Technical Risk (QTR) Project. 

Table 5.3-3. Quantitative Risk Projects 


Projects 


Quantitative 
Technical Risk 


Started Purpose 


Develop a mix of near-term and long-term MPTs that quantify the 
2012 technical risk programs face to support improved decision making 
about how to address those risks 


Primary 

SEMT 

Supported 

Strategies 


5.3A1 Quantitative technical Risk (Q1R) Project 

Table 5.3-4 shows the focus, deliverables, and investment in the QTR Project through 2014. 
The timeline beyond 2014 has not yet been established, and will be structured to build on 
progress in 2013 and 2014. Technical risk refers to risks that originate in, have effects on, or 
involve in risk mitigation including system configuration, architecture, baseline, technologies, 
design, manufacturing and/or integration. The QTR Project has two primary foci, to be 
addressed in parallel. 
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The first focus is incremental development of QRT MPTs addressing needs and gaps within the 
DoD acquisition Risk Management process, starting with "low-hanging fruit" then progressing 
to more challenging issues. The approach is to involve end-users in the services as co¬ 
developers and transition partners in order to ensure that the QTR are relevant to program risk 
issues; practical within the acquisition decision and information structures, processes, and 
temporal sequences; and can be effectively transitioned to end-users. The intent is to employ 
evidence-based methods to identify high-impact and high-contribution MPTs. 

The second focus is fundamental research needed to develop QTR MPTs addressing challenging 
DoD Risk Management needs and issues. The approach is to adapt and expand promising 
theoretical frameworks and MPT approaches from areas outside of traditional DoD Risk 
Management, such as insurance underwriting for large-scale engineering and construction 
projects, real options in product design and development, and predictive analytics in insurance 
and business development. The scope includes risks over the entire system life cycle, 
peculiarities by type of system, type of acquisition, type of cause, type of causal chain, and 
types of mitigation strategies. 


Table 5.3-4. Quantitative Technical Risk Project Timeline 


Year Focus Key Deliverables 


Framing the issues, opportunities and 
technical approaches for relevant, 
Pre-2014 practical QTR MPT near- and long-term 
research, development, validation and 
transition 


Research framework of near- and long-term risk issues 
& potential QTR MPT for DoD acquisition. Findings 
from investigation of complexity theory for acquisition 
program risk management. List of approaches from 
outside traditional DoD acquisition. Plan for 
incremental development, validation and transition. 
Identification of potential case study systems and data 
sources. Outlines of strategies toward developing an 
adaptable integrated risk management framework, and 
transitioning MPT to end-user systems. 
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2014 


Incremental development, validation 
and transition of QTR MPT in parallel 
with longer-term research for 
additional QTR MPT. 


Data for one or more historical systems and acquisition 
programs for use in case studies. Documentation of 
one or more initial QTR MPT harvesting "low-hanging- 
fruit", with case studies applying the initial QTR MTP to 
a historical acquisition program. Letters of agreement 
with end-user co-development and transition partners 
in one or more services. Technical interface 
specifications and coordination milestones toward 
transition initial QTR MPT to a service SE and risk 
management system. Identification and assessment of 
"flexibility and adaptability" strategies and risk-hedges 
(real option) in DoD acquisition. Identification and 
assessment of Risk Breakdown Structures, Risk Factor 
Analysis and Risk Estimating Relationships (for 
predictive analytics) for DoD programs. Assessment of 
insurance industry approaches to advanced engineering 
projects for DoD acquisition programs. 


2015-2018 


Incremental development, validation 
and transition of QTR MPT in parallel 
with longer-term research for 
additional QTR MPT. 


Documentation of incremental QTR MPT, case study 
validation, and transition to end-users SE and risk 
management systems. Further research on underlying 
methods and theory. Specific topics to be determined. 


5.3.5 Interactive Model-Centric Systems Engineering (IMCSE) Program 

The IMCSE Program will include and significantly extend the traditional focus on the modeling 
of system products and the use of the models to perform ilities tradespace and affordability 
analyses as described above, and increasingly use the models to generate software and 
hardware products. The IMCSE Program extensions will address the modeling of system 
execution processes, such as operational concept formulation, and system development 
processes, which can also be executed to aid in the generation of system products. Further, as 
was emphasized in the IMCSE section of the SERC Systems 2020 Report, an additional focus on 
modeling the system's environment will be pursued, which is needed for performing many of 
the ilities tradespace and affordability analyses. Models can also improve affordability by 
automatically generating needed documentation, or even better by serving as the 
documentation itself. Further, models can reduce or avoid system overruns and performance 
shortfalls by enabling more thorough Analyses of Alternatives and evidence-based decision 
reviews. 

This research program primarily implements all four of the SEMT strategies noted above. It 
builds on a recently completed set of projects (RT-30: Graphical Concept of Operations), and the 
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first IMCSE project currently under definition that will begin in late 2013. Table 5.3-5 offers a 
description of the active project and the strategies it primarily supports. 

Table 5.3-5. IMCSE Projects 




Primary 

Projects 

Started Purpose 

SEMT 

Supported 



Strategies 

Interactive Model-Centric 

Using models to drive systems engineering, 

Late 2013 . . , , . 

development, production, and evolution 

1, 2, 3, 4 

Systems Engineering 


5.3.5.1 Interactive Model-Centric Systems Engineering Project 

Models have significantly changed SE practice over the past decade. Most notably, model- 
based systems engineering (MBSE) methods and tools are increasingly used throughout the 
entire system lifecycle to generate systems, software and hardware products, replacing labor- 
intensive and error-prone documentation-based processes with model-based ones. While 
substantial benefits have been achieved, the most impactful application of models in SE has yet 
to be realized. Truly transformative results will only come through intense human-model 
interaction, to rapidly conceive of systems and interact with models in order to make rapid 
trades to decide on what is most effective given present knowledge and future uncertainties, as 
well as what is practical given resources and constraints. The Interactive Model-Centric 
Systems Engineering (IMCSE) research program seeks to achieve this transformation. 

The complexity and socio-technical nature of contemporary systems/SoSs drives an urgent 
need for a more powerful integration of humans and technologies. Early concept decisions 
have always been critically important, and with continuously evolving SoSs having long life 
spans, such decisions are now made throughout the entire life cycle. Soft factors become 
increasingly influential. For example, trust in model-based data sets and decisions are in part 
determined by the chosen model itself as perceived by specific decision makers. The timescale 
of making early architectural decisions is out of sync with the current model-based systems 
engineering capabilities and decision environments. New algorithms and novel modeling 
approaches must be discovered to accelerate technical and programmatic decision support 
from months to minutes; in order to effectively leverage and incorporate human knowledge 
and judgment, this requires an interactive capability. Much potential exists in maturing 
emerging novel methods for evaluating system responsiveness under complex uncertainties, to 
enable engineering of resilient systems. Further, as was emphasized in the SERC Systems 2020 
Report, modeling the system's dynamic operational environment remains an open area of 
research. Forward research is informed by a recently completed set of SERC projects on 
Graphical Concept of Operations. 


Approved October 25, 2013 


33 


SERC 2014-2018 Technic a I Plan 






Unlimited Distribution 


The IMCSE research program will involve three projects initiated in 2014, followed by additional 
projects in out-years as determined by the 2014 investigation of current state of art/practice. 
The three projects are: 

1. Pathfinder Project. The project will investigate the current state of the art/practice in 
interactive model-centric systems engineering. Surveys and literature review will be 
used to establish a preliminary picture of what is being done in practice, current MPTs, 
and what research has/is being performed. This will inform the planning and conduct of 
an invited workshop to identify research opportunities, gaps and issues. A report will be 
developed from the workshop, including priorities for expanded and additional research 
in out-years of the project. 

2. Interactive Schedule Reduction Model. This project will build on an existing prototype 
model (prior DARPA-support) for interactively exploring alternatives in the systems 
development process and application of resources. The model enables rapid sensitivity 
analysis of various factors to determine their potential impact on program schedule. 
Exploratory extensions of the model will be developed and evaluated, resulting in a new 
prototype for pilot application. A report will be developed on the findings and plans for 
pilot testing. 

3. Interactive Epoch-Era Analysis. This project will involve research to extend a current 
approach for evaluating systems under uncertainty, Epoch-Era Analysis (EEA), through 
the development of an interactive capability. The resulting prototype method and 
supporting tools will be applied to a case on uncertainties in mission planning and 
deployment support, of particular interest to the ERS program. This case application will 
serve as a pathfinder for identifying key considerations for applicability and 
deployability of the method for eventual DoD use. A report will be developed on the 
findings for the research and case application, along with plans for further development 
and case applications. 


Table 5.3-6 shows the focus, deliverables and investment in IMCSE through 2018 aimed at 
addressing the three projects above, as well as new projects in out-years. In 2014, the 
"pathfinder" project investigation will identify synergistic research opportunities. Beginning 
with the initial 2014 workshop, periodic targeted workshops will be convened with the intent of 
examining ongoing research that can be leveraged in the SERC research efforts, and involving 
the broader community in creating - and realizing - the vision and research agenda for the 
IMCSE program. 


Approved October 25, 2013 


34 


SERC 2014-2018 Technic a I Plan 



Unlimited Distribution 


Table 5.3-6. IMCSE Project Timeline 


Year Focus Key Deliverables 


Pre-2014 New start 


Pathfinder project with collaborative 
research discovery; exploratory 

2014 extensions to an existing development 

schedule reduction model; exploratory 
piloting and interactive extensions to 
Epoch-Era method 

Pathfinder Project: Investigation of current state of 
art/practice. Workshop to explore issues and 
opportunities, with report on workshop results. 

Pathfinder project report, with findings of research 
opportunities, gaps, and issues. Out-year research 
plans based on pathfinder results. 

Interactive Schedule Reduction Model: Exploratory 
extensions implemented and evaluated. Report on 
exploratory schedule model. Prototype model for pilot 
application. 

Interactive Epoch-Era Analysis: Exploratory research to 
develop interactive capability, with demonstration via a 
mission planning support application case. Report on 
exploratory research and case application. 

Initiate multi-year research plans based 
on pathfinder results, including 2014 
2015 project follow-on for one or both of the 

exploratory research projects. Assess 
results individually and comparatively. 

IMCSE Project Applications: Based on pathfinder project 
results, select and initiate one or more additional 
projects, and increase SERC member collaboration in 
projects. Report to document the maturation of the 

MPTs for each of these projects, with comparative 

results. 

Increasing maturation of IMCSE MPTs 
and enabling environments, leading to 
adoption by user community and 
assessment of real-world impact; 
extend IMCSE scope via increased 

2016 

collaboration of additional universities 
and broader user community. 

Exploration of further new-idea 
projects. 

IMCSE MPT Implementations and Impact Assessments: 
Continued maturation and implementation of IMCSE 
MPTs, with enabling environments. Ongoing study of 
impacts resulting in a comprehensive report of 
progress, results, and opportunities. 

Increasing maturation and synthesis of 

IMCSE MPTs and enabling 

environments, leading to adoption by 

user community and demonstration of 
2017-2018 , , , 

real-world impact; sustain and increase 

collaboration of additional universities 

IMCSE MPT Synthesis Impact and Effective Practice 
Assessments: Continued maturation, synthesis and 
implementation of IMCSE MPTs, with enabling 
environments. Ongoing study of real-world impacts to 
identify successful practices. A comprehensive report of 


and broader user community. Step-ups impacts and insights, with guidance on practice, 
of new-idea projects. 
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5.3.6 Agile Systems Engineering Program 

Agility can be defined as the ability to respond quickly and effectively to changes. In SE, there 
are two general areas where agility is a primary characteristic - process and product. Product 
agility through better architecting has been addressed extensively in SERC RT-18 on Valuing 
Flexibility, and in earlier work such as the Rechtin-Maier series of Systems Architecting books, 
the CMU-SEI Architecture Tradeoff Analysis Method book and reports, and the DARPA F6 and 
META projects. This effort will primarily focus on agile and lean process approaches. 

Process agility provides systems engineers with methods, processes and tools to operate more 
effectively in development environments driven by change: the rapid pace of technology, the 
increasing need for interoperability between legacy and new capabilities, evolving 
requirements throughout the development lifecycle, and the changing economic and political 
factors that undergird and enable system development. 

A key consideration in both process and product agility is the reconciliation and integration of 
systems and software engineering activities. Software by its nature is malleable and able to 
rapidly achieve modified or new capabilities while maintaining system operational capability. 
Currently, software development processes do not operate seamlessly with systems 
engineering processes. The ability for software to provide incremental capability requires 
adaptation of SE MPTs to enable modularity, flexibility and continuous integration and 
verification and validation (V&V). 

The Agile SE Program currently has two projects: 

1. Kanban in SE 

2. Agile SE Enablers and Quantification 

Table 5.3-7 offers a description of the two active projects and the strategies they primarily 
support. 


Table 5.3-7. Agile SE Projects 


Project 

Started 

Purpose 

Primary SEMT 
Supported 
Strategies 

Kanban in SE 

2010 

Exploration of applicability of Kanban methods to complex system 
evolution where development is primarily through capability 
enhancement and new features added to an existing system and 
infrastructure 

2,3,4 

Agile SE 
Enablers and 
Quantification 

2008, 

2012 

Identifying and employing research that can inform and improve 
agile SE, drawing on the SERC RT-34 Expediting SE study and its 
initial model quantifying agile schedule compression 

1,2,3,4 
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5.3.6.1 Kanban in SE Project 

Kanban in SE explores the use of lean scheduling methods that have been successful in 
manufacturing. "Pull" or "on demand" scheduling methods smooth workflow by limiting 
scheduled work to current resource capacity and constantly reprioritizing the work to be 
performed according to real-time status information. This promotes earlier delivery of work 
and higher value to the downstream customer, improves the efficiency of scarce resources, 
prevents overloading of resources, and limits the overhead of maintaining constantly changing 
schedules. 

This research was initially motivated by the ineffectiveness of integrated master schedules in 
rapidly changing operational environments and the success of Kanban approaches for the 
knowledge work of software development. Research to date has shown that Kanban 
approaches can be successful in the development and evolution of individual systems. Further 
research is necessary to determine the extent to which Kanban methods can be successfully 
applied in the initial definition of highly complex systems, or in the creation, operation, and 
management of systems of large, independently evolving systems. 

The proposed research builds on the initial Kanban scheduling system (KSS) networks described 
in SERC RT-35. The KSS network seeks to prioritize engineering tasks based on SoS or complex 
system capability priorities and task interdependencies. This leads to implementation of the 
most value-adding features first, reduces wait time for scarce engineering specialties, and 
minimizes time wasted on context switching by overloaded resources. In achieving this goal, a 
KSS network provides two valuable side effects. First, the implementation of the network 
supports critical conversations about schedule and value decisions by the appropriate people at 
the right time and nearest the actual implementation. Second, the network significantly 
improves executive and systems engineering visibility into the status of multiple independent 
development organizations. 

The proposed next steps in this research are to improve the simulation and analysis 
infrastructure and conduct industry pilots of the KSS network concepts to evaluate and 
calibrate the infrastructure. 

The research infrastructure developed in RT-35 is insufficient for further research on the 
practicality and implementation of KSS networks. Current simulation capabilities need to be 
expanded to go beyond the comparison with traditional scheduling systems and provide the 
capability to model different organizational structures and to provide sensitivity analysis of the 
various flow control methods. The proposed Lean Kanban work will expand the basic model to 
include a fuller set of simulation features and reports. Several software and SE infrastructure 
tool manufacturers have confirmed interest in supporting this work using their own tool 
infrastructure. 
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As the analysis instrumentation evolves, in vivo piloting of the concept in industry will evaluate 
and calibrate the models. Several aerospace companies have agreed to support the piloting 
effort. By simulating the KSS network and comparing the results to actual pilots, actual schedule 
and effort savings can be captured and then extended through further modeling and simulation 
to various types of complex system developments. In addition, the results will provide the 
information necessary to justify an expected return on investment for these types of process 
changes. Finally, the results from simulation scenarios and actual pilots can be used as 
feasibility evidence for new system/system capability cost and schedule estimates, identify 
those estimates that are overly optimistic or overly conservative, and provide insights on the 
general range of applicability of Kanban approaches. 

This work will take advantage of a large, skilled volunteer working group of industry 
professionals created during the RT-35 task. Members of the group represent the companies 
interested in both the infrastructure and piloting support. Table 5.3-8 shows the focus, 
deliverables, and investment in Kanban through 2018. 


Table 5.3-8. Kanban in SE Project Timeline 


Year 

Focus 

Key Deliverables 

Pre-2014 

Studying on-demand scheduling 
systems and adapting the concept 
to SE 

Kanban-based Scheduling System (KSS) concepts; RT-35 
Kanban in SE reports and papers; Kanban in SE industry 
working group 

2014-2015 

Simulating, and piloting KSS 
networks 

Demonstration, deployment and evaluation materials; 
piloting plans; piloting results 

2016 

Enhancing KSS Concept 

Continued piloting; Adding economic factors; adding 
Bayesian Belief Network supported estimation based on 
Kanban system information and flow data to support 
capability selection and staffing decisions 

2017-2018 

Extending KSS concept 

Adapt the KSS concept to additional SE environments 


S3.6.2 Agile SE Enableisand Quantification Project 

Candidate Agile SE enablers include the product, process, people, project, and risk factors 
identified in the SERC RT-34 Expediting SE study, and the candidate enablers identified in RT-35: 
Life cycle process selection (ICSM, mixed-mode); Human cognition, negotiation and 
communications; Governance of evolving system development; and Organizational and system 
goal alignment. 

This task draws on the extensive interviews, site visits, and analyses of expedited SE experience, 
and the associated SE acceleration estimation model developed in the SERC RT-34 Expedited SE 
study. It can also take advantage of a number of activities external to SERC, including the 
INCOSE Agile SE Working Group, the Lean Systems Society and its projects, and the 
programmatic information from MITRE and other FFRDCs. 
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Table 5.3-9 shows the focus, deliverables, and investment in Agile SE Enablers through 2018. 


Table 5.3-9. Agile SE Enablers Project Timeline 


Year 

Focus 

Key Deliverables 

Pre-2014 

MPTs in software 

MPT reports and papers; RT-35 phase 1 work; RT-34 work; 
support of INCOSE Agile SE WG 

2014 

Fundamentals of agility 

White papers on topics such as: SE as a service concept; 
Complexity and sense-making in SE decisions; 

Governance, organizational and contracting requirements 
for implementing agile SE. Quantitative schedule 
acceleration model extensions, calibration, behavioral 
analysis. Experimental applications and resulting 
formulation of further high-impact Agile SE research. 

2015-2018 


Performance, evaluation, and refinement of further high- 
impact Agile SE research project results 


5.4 Human Capital Development (HCD) 

Over the last decade, DoD and the defense industrial base (DIB) have often cited a shortfall in 
the quantity of systems engineers and in the knowledge, skills, and abilities (KSAs) of those 
systems engineers (NDIA 2006, WSARA 2009, NDIA 2010). The Human Capital Development 
(HCD) research area directly targets that shortfall. 

Goal: Ensure a competitive advantage through the availability to the DoD and the defense 
industrial base of highly capable systems engineers and technical leaders 


5.4.1 HCD Grand Challenge 

The HCD grand challenge to achieve the HCD goal is to: 

Discover how to dramatically accelerate the professional development of highly 
capable systems engineers and technical leaders in DoD and the defense 
industrial base and determine how to sustainably implement those findings 


5.4.2 Strategy to Address the HCD Grand Challenge 

Successfully executing the following strategies will make significant progress towards 
addressing the HCD Grand Challenge: 

1. Create and Provide Easy Knowledge Access: Make it easy for systems engineers to 
understand the SE discipline and to access the information needed to expertly perform 
SE so that the workforce can master the most important competencies 
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2. Educate and Train Faster: Develop innovative approaches and technology to educate 
and train systems engineers and systems teams at all levels, engineers, and STEM 
students much more rapidly, effectively and efficiently than with classical means 

3. Develop Effective Technical Leaders: Develop innovative approaches to educate DoD 
technical leaders with the right mix of technical, business, and enterprise skills 

4. Improve SE and STEM Education: Develop recommendations and systems curricula for 
the next generation of systems engineers, engineers and STEM students 

5. Track Progress: Track the changes in SE workforce demographics and performance over 
time to understand how the workforce is improving and how improvement programs 
are working 

Three HCD research programs directly implement the strategy: 

• Evolving Body of Knowledge 

• Experience Acceleration 

• Systems Engineering and Technical Leadership Education 


The synergy and interaction of the HCD programs and projects is shown in Figure 5.4-1 below. 
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Figure 5.4-1. HCD Program and Project Synergy to Meet Grand Challenge 
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5.4.3 Evolving Body of Knowledge Program 

This research program primarily implements HCD strategies 1, 4 and 5 above - Create and 
Provide Easy Knowledge Access, Improve SE and STEM Education, and Track Progress. It 
includes two projects already underway (BKCASE and Helix) and a third project (SEEK) that will 
begin in late 2013. Table 5.4-1 offers a description of these three projects and which strategies 
they primarily support. 


Table 5.4-1. Evolving Body of Knowledge Projects 





Primary HCD 

Projects 

Started 

Purpose 

Supported 

Strategies 

BKCASE 

2009 

Create, disseminate, and evolve the authoritative guide to the SE body of 
knowledge and recommendations on curricula for SE graduate programs 1 

1, 3, 4, 5 

HELIX 

2012 

Understand and widely disseminate the characteristics of the SE 
workforce, including what enables them to be effective at their jobs 

3, 5 



Capture insights from SE experts about significant program successes and 


SEEK 

2013 

failures and portray them primarily in case studies augmented by multi- 
media materials for easy access and use in training, education and practice 

1,3,4 


5A3.1 BKCASE Project 

The Body of Knowledge and Curriculum to Advance Systems Engineering (BKCASE) Project is (a) 
identifying and making readily accessible the vast knowledge that systems engineers need to 
know (SEBoK) and (b) providing recommendations to the SE academic community on SE 
graduate curricula (GRCSE). BKCASE began in 2009 as a SERC project led by Stevens Institute 
and the Naval Postgraduate School. Beginning in 2013, the International Council on Systems 
Engineering (INCOSE) and the Institute of Electrical and Electronics Engineers Computer Society 
(IEEE-CS) have become co-stewards with the SERC to guide and promulgate the SEBoK and 
GRCSE. Both products will undergo regular updates to reflect advances in the field and 
feedback from the user community. SEBoK articles have been accessed well over 100,000 times 
since Version 1.0 was released in September 2012. Several universities in the US, Europe, and 
Australia have begun adopting GRCSE curriculum recommendations. Table 5.4-2 shows the 
focus, deliverables, and investment in BKCASE through 2018. The SEBoK is novel in its form of 
delivery (a wiki), its governance model (shared among 3 organizations), its scale (spanning the 
technical aspects of the discipline, how that technology is effectively adopted and used, and the 
underlying science on which the technology is based), and its rate of change (multiple updates 


1 SEBoK is naturally aligned with the Evolving Body of Knowledge Program. GRCSE is naturally aligned with the SE 
and Technical Leadership Education Program. However, for historical reasons, BKCASE produces both SEBoK and 
GRCSE. Since SEBoK is far more demanding than GRCSE to create, promulgate, and evolve, BKCASE is 
categorized within the Evolving Body of Knowledge Program. 
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annually). Research in how to maintain this novelty and how to tailor the SEBoK to specific 
domains is the primary research focus for the next several years. 


Table 5.4-2. BKCASE Project Timeline 


Year 

Focus 

Key Deliverables 

Pre-2014 

Early versions of SEBoK and GRCSE, 
governance model to promulgate and 
support them 

SEBoK 1.0,1.1,1.2, GRCSE 1.0, governance agreement 
between SERC, INCOSE, IEEE-CS, workshops 


Adoption, new partnerships, expanded 

Agreements with other societies, domain-specific 

2014 

content, broader community 

variants of SEBoK, SEBoK 1.3,1.4, GRCSE 1.1, user 


contributions 

groups, adoption workshops 


Continued adoption, partnerships, 

Additional society agreements and domain-specific 

2015 

expanded content, broader community 

SEBoK variants, SEBoK 2.0, GRCSE 1.2, user groups, 


contributions 

adoption workshops 

2016-2018 

Broad community use 

SEBoK 2.1, 2.2, GRCSE 1.3, user groups 


5A3.2 Helix Reject 

Helix began in October 2012 to examine the "DNA" of the systems engineering workforce in 
both DoD and the defense industrial base. The project will answer three questions: 

• What is the demographic picture of the systems engineering workforce? What is the 
distribution of people by age, education, title, role, activity, etc.? How does the 
distribution of people differ based on organizational mission, size, etc.? 

• How effective are systems engineers and why? What are the forces most impacting 
effectiveness - experiences, proficiency in specific competencies, culture of the 
organization in which they work, etc.? 

• What are employers doing to improve the effectiveness of the systems engineering 
workforce and how well are those efforts working? Are they using mentoring, job 
rotation, in-house training, etc. and how aligned are those efforts to the forces that 
most impact effectiveness of their workforce? 

As more data is collected and analyzed, regular reports of aggregated anonymous data will 
provide an increasingly rich and varied set of insights into the systems engineering population. 

Table 5.4-3 shows the focus, deliverables, and investment in the Helix Project through 2018. 

Table 5.4-3. Helix Project Timeline 


Year Focus Key Deliverables 


Establish methodology, small-scale data Early findings report from both DoD and the defense 
collection, publish early findings industrial base 
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Year 

Focus 

Key Deliverables 


Large-scale data collection and analysis; 


2014 

begin regular publication of findings; 
possible expansion to non-defense and 
non-US organizations 

Quarterly reports of findings; benchmarking studies 

2015-2018 

Data collected from enough 
organizations and individuals to paint a 
vivid and insightful image of the systems 
engineering population 

Quarterly reports of findings, including changes in 
findings overtime based on longitudinal data 
collection; benchmarking studies 


5A3.3 SEEK Project 

This project addresses a gap in the SE research literature: the lack of detailed case studies about 
SE successes and failures. The research will develop a series of case studies tailored to defense 
education needs. These case studies will support instruction at the Defense Acquisition 
University and at the Naval Postgraduate School, the federal service academies, and other 
government education and training providers. They will be managed by a board of advisors 
from stakeholders to increase the utility and portability of the case studies. They will provide 
core data for other SE research. 

Much SE corporate knowledge resides in engineers nearing retirement. Those engineers have 
presided over the successes and failures of the last decade; they know the lessons learned and 
the technical details of what worked and what has not worked. This project will capture that 
knowledge. Case studies allow the community to capture the technical data of the projects, 
understand what worked and what did not, and then to generalize. They are invaluable for 
process improvement across Do/D and for education that is grounded in practical examples. 

Table 5.4-4 shows the focus, deliverables, and investment in the SEEK Project through 2018. 


Table 5.4-4. SEEK Project Timeline 


Year 

Focus 

Key Deliverables 

Pre-2014 

Form project and set up infrastructure 
to capture expert knowledge 

None 



Several substantial case studies annually, integrated 


Grow pool of experts; conduct 

with the SEBoK, Defense Acquisition Guidebook, and 

2014-2018 

interviews and roundtable discussions; 

INCOSE Handbook. The number will scale with funding, 


develop case studies 

and the topics will be selected based on advisory board 
input. 


5.4.4 Experience Acceleration Program 

This research program primarily implements HCD strategy 2 - Educate and Train Faster. It will 
include projects aimed at creating automated learning environments that simulate real world 
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experiences of systems engineers. Those experiences will be vivid and realistic enough to 
significantly accelerate the learning and maturation of those systems engineers. One project 
will evolve the current simulation platform, making it ever more robust and capable and 
enabling quicker and easier construction of new experiences. Other projects will add to the 
current catalog of experiences, developing new experiences that use the simulation platform. 
Experiences will vary based on the size and types of systems being acquired, the acquisition 
lifecycle, the novelty of the technology being acquired, and other parameters of interest. Over 
the five-year period from 2014-2018, other organizations will join the SERC in improving the 
experience platform and in developing additional experiences, creating a marketplace for 
experience acceleration. Table 5.4-5 offers a description of these projects and which strategies 
they primarily support. Table 5.4-6 shows the focus, deliverables, and investment in the 
Experience Accelerator Project through 2018. 


Table 5.4-5. Experience Acceleration Projects 


Projects Started 


Experience 

Accelerator 


Additional 

Virtual 

Experiences 


2010 


2014 


Purpose 


Create the "engine" that will be used to host a wide range of 
experiences, develop the first virtual experiences that use the engine, 
and validate the experience accelerator concept through trial use. 

Keep improving the engine over time as a broader set of experiences 
are created and trialed with ever more students. Create an open 
vibrant community that will develop additional virtual experiences that 
can be shared within the defense industrial base and DoD. 

Develop an increasingly broader and richer set of virtual experiences 
that are hosted on the Experience Accelerator engine 


Primary 

HCD 

Supported 

Strategies 


Table 5.4-6. Experience Accelerator Project Timeline 


Year 

Focus 

Key Deliverables 

Pre-2014 

Build experience accelerator engine, 
first experiences, and validate 
concept with first students 

Increasingly sophisticated versions of Experience 
Accelerator engine, first virtual experience, first use by 
students 

2014-2018 

Create an increasingly more 
complete and sophisticated engine, 
tools to build virtual experiences, 
and a marketplace community to 
create and apply those experiences 

New versions of increasingly more capable engine and 
tools to create experiences, marketplace community with 
suitable governance structure 
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5.4.5 Systems Engineering and Technical Leadership Education Program 

This research program primarily implements HCD strategies 2 and 3 - Educate and Train Faster 
and Develop Effective Technical Leaders. It will include two projects aimed at improving the 
quality of education in systems thinking and SE in universities that teach engineers who are 
hired by the DoD and the defense industrial base: the Engineering Capstone Marketplace 
Project, and the Technical Leadership Project. Table 5.4-7 offers a description of these projects 
and which strategies they primarily support. 


Table 5.4-7. Systems Engineering and Technical Leadership Education Projects 


Projects 

Started 

Purpose 

Primary 

HCD 

Supported 

Strategies 

Engineering 

Capstone 

Marketplace 

2010 

Originally intended to show how to conduct multidisciplinary senior 
capstone projects in classical engineering programs, especially those 
that increase awareness of and appreciation for DoD applications. 
Building on its early success, the project has morphed into a 
marketplace where companies and government organizations post on 
a website problems suitable for senior capstone projects. Students 
from multiple universities form teams to work on those projects 
under the supervision of faculty and the posting organization. This 
marketplace model has the potential to scale nationwide, involving 
thousands of students in hundreds of projects and universities. 

2,4 

Technical 

Leadership 

2010 

Develop innovative ways to teach technical leadership to the DoD 
acquisition workforce, including not only systems engineers, but also 
others who must understand technical leadership, such as program 
managers. Iteratively pilot the resulting courses and integrate them 
into the DAU curriculum. 

2,3,4 


5.45.1 Engineering Capstone Marketplace Project 

The Engineering Capstone Marketplace Project is the evolution of research begun in 2010, 
which showed that a multidisciplinary senior capstone project could enhance development of 
SE competencies and increase interest in SE. The Marketplace Project is now building and 
piloting the infrastructure to affordably scale this approach nationwide between 2014 and 2018 
and improve how thousands of students are taught engineering across the US. Sponsoring 
organizations contributing problems should get innovative solutions developed by students 
who come to appreciate and understand how to work in multidisciplinary teams that employ 
systems thinking and systems engineering. Universities should substantially increase the value 
that their senior capstone projects offer to students. Problems contributed by the defense 
industrial base and DoD will produce students who are more aware of and appreciative of 
national security issues, and are more likely to seek national security employment. Table 5.4-8 
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shows the focus, deliverables, and investment in the Engineering Capstone Marketplace Project 
through 2018. 


Table 5.4-8. Engineering Capstone Marketplace Project Timeline 


Develop and validate on a small scale 
approaches to fostering 

Pre-2014 multidisciplinary senior capstone 
projects both within single 
universities and across universities 


Key Deliverables 


Reports on how to foster multidisciplinary senior 
capstone projects, experience reports from participating 
universities, infrastructure to support nationwide 
expansion of the approach 


2014-2018 


Expand the Marketplace to draw 
hundreds of students from all across 
the country and across many 
universities, industrial and 
government organizations to sponsor 
multidisciplinary capstone problems 


Robust infrastructure capable of supporting large-scale 
involvement of universities, students, and organizations, 
experience reports, outreach mechanisms to engage a 
nationwide audience of students, universities, and 
organizations, analysis of the value and impact of the 
Marketplace 


5.45.2Technical Leadership Ftoject 

The Technical Leadership Project began in 2010, creating an innovative approach to educating 
technical leaders through three lenses: systems, business, and enterprise. A series of three five- 
day courses have been prototyped and piloted in 2012 and 2013. Each course contains a series 
of independent readings, lectures, case studies, and student in-class exercises to accelerate 
systems technical leadership learning. The courses take the student from (a) leading systems 
development in the face of uncertainty and ambiguity to (b) understanding how commercial 
businesses or organizations accountable for multi-system and multi-customers strategize, 
operate and measure performance to (c) the technical leadership expectations of an enterprise 
senior technical leader responsible for assessing and adapting multi-nodal structural and 
activity-based processes within DoD or commercial enterprises. Additional piloting will take 
place into 2014 when those courses will begin to be integrated into the DAU curriculum. Table 
5.4-9 shows the focus, deliverables, and investment in the Engineering Capstone Marketplace 
Project through 2015, at which time it is likely to conclude. 


Table 5.4-9. Systems Engineering and Technical Leadership Education Projects Timeline 


Year 

Focus 

Key Deliverables 


Create and pilot three courses for 


Pre-2014 

the systems, business, and 
enterprise lenses 

Course materials and reports from pilot classes 


Mature the courses and integrate 
them into the DAU curriculum 

New versions of course materials for standalone pilots and 

2014-2015 

for integration into the DAU curriculum; reports from pilot 

classes 
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5.5 Supporting Activities 


5.5.1 New Project Incubation 

While it is believed that that the aforementioned research programs have a great potential to 
have a transformative impact on the DoD and 1C, there is a need to support new ideas in their 
infancy that may become the critical research programs for emerging challenges. This 
incubation capability will be supported by an annual open call to the SERC research 
collaborating universities to propose early stage research that can be nurtured through 
relatively small levels of seed funding. Preference will be given to proposals that contend with 
issues not currently being addressed by SERC research, or that use novel approaches. The 
Sponsors, SERC Research Council and Leadership will make the determination of which 
proposals will be awarded with this seed funding. The SERC incubator timeline is shown in Table 
5.5-1. 


Table 5.5-1. SERC Project Incubation Timeline 


Year 

Focus 

Key Deliverables 


Create proposal format and 


Pre-2014 

process in consultation with SERC 
sponsors with input from SERC 

collaborators 

Announcement of incubator proposal solicitation to SERC 
collaborating universities 

2014-2018 

Selection of top proposals for 
funding 

Award of seed funding to selected projects 


5.5.2 Accelerated Development and Transition 

Along with direct funding in each of these areas, incentives will be put in place to motivate Pis 
and researchers to find funding for their programs and projects outside of the sponsorship. 
These incentives come both from a requirement for the PI to create funding plans when they 
create or respond to a proposal, and matching funding for the initial external funding. 

There are also incentives for the transition of the research into practice. As with external 
funding, while each RT is required to have transition plans in place in the proposal prior to 
award, additional funding will be provided as an incentive for this work. In addition, funds will 
be allocated for the development of educational materials and courses, to be shared by all SERC 
collaborating institutions, based on research results. The SERC accelerated development and 
transition timeline is shown in Table 5.5-2. 
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Table 5.5-2. SERC Project Incubation Timeline 


Year 

Focus 

Key Deliverables 


Create and document proposal 
requirements for external funding 


Pre-2014 

and transition plans in 
consultation with SERC sponsors 
with input from SERC 
collaborators. Establish policies 
for matching, transition and 
education material creation funds. 

Announcement of external funding and transition plan 
requirements for proposals to SERC collaborating 
universities 

2014-2018 

Selection of research for matching, 
transition and education material 
creation funding 

Award of matching, transition and education material 
funding 


5.5.3 Infrastructure Development 

Note that the SERC Vision after completion of this 5-Year plan is that: 

• The SERC operates the world's largest and most-visited SE Research Web site, including 
the largest and best-organized SE Research experience base. 

• It continues to provide leadership in evolving the SE Body of Knowledge. 

• It runs the most widely-attended and highest-rated SE Webcast series. 

Focus is needed to develop the infrastructure that can be used by all of the research programs 
to achieve these goals. Our approach is to: 

• create a list of primary IT services through a comprehensive functional decomposition 

• define the resultant requirements through specific capabilities 

• determine cost and time to create desired capabilities 

• prioritize and schedule development 

A set of potential set of SERC IT services are shown in Figure 5.5-1. These services are classified 
by those which are available to all who access the SERC website, those which are available to all 
SERC collaborators, and finally, those which are used internally by SERC staff. 


Approved October 25, 2013 


48 


SERC 2014-2018 Technic a I Plan 









Unlimited Distribution 


r 


SERC IT Services 





■■1AIIHH1 

Collaborators 


Internal 



General 
Information 
and Services 


(moved to Doc & 
Pubs) 


Knowledge 

Management 


Enterprise Search 


Knowledge 

Capture 


Knowledge 

Discovery 


Advanced 

Knuwlrdgr 

exploration 


Media & 
Publications 


Media 

Repository 


Fill .ironic 
Proceedings 


Phy.M.il Pnxiuih 


Brokered Products 


Learning 

Management 


Coll.ibor.itcM 


Doctoral 


Programs 


so If A'.'.r'.'.rrvnt'. 



Couc.r Crr.ition 


Course 

MiMWgrmrr 


Event Support 


CSER 


SSRR 


Rl-M-.MCh 

workshops 


Collaborator 

Events 


RT Group Events 


Collaborator 

webiiun 



External Events 


Research 

Transition 

Support 


ProblermS 

Research 

Marketplace 


Community 

Resources 


RT. Generic Sites 


Discussion Forums 


Non-Member 

Mfmt 


Social Media 
Integration 


Collaborator, 

Sponsor, 

Partner 

Management 


sf-RC Doctoral 
Fellows Program 


Transaction 

HiMoiy 


Financial & 
Contractual 
Transactions 


Supporting Infrastructure 


Figure 5.5-1. Potential SERC IT Services 

This work will be scoped to be accomplished through SERC investment funding and will not 
require core funding. The SERC IT Services timeline is shown in Table 5.5-3. 


Table 5.5-3. SERC IT Services Timeline 


Year 

Focus 

Key Deliverables 

Pre-2014 

IT Services project plan 

Prioritized IT Service list and plan 

2014-2018 

Development of IT Services 

Phased delivery of IT services 
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